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6 Abstract 


This  report  documents  the  results  of  a study  to  assess  the  impact  of  the  Automated 
Radar  Terminal  System  (ARTS  $JI)  upon  air  traffic  control  system  capacity.  It  is 
based  upon  on-site  measurements  of  air  traffic  activity,  controller  work  pace, 
controller  workload  indicators,  and  staffing  at  the  San  Antonio,  Texas  and 
San  Francisco/Oakland,  California  TRAC0NS.  Measurements  were  taken  of  both  the 
pre-ARTS  1^  and  post-ARTS  environments  and  comparisons  made  to  determine  the 
direction  and  degree  of  change  attributable  to  ARTS  ijfr.  The  results  indicate  that 
the  ARTS  r^I  system  has  reduced  controller  workload  and  increased  system  capacity 
by  10.52.  A reasonable  estimate  of  the  productivity  increase  is  8.52. 
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METRIC  CONVERSION  FACTORS 


EXECUTIVE  SUMMARY 


This  report  documents  the  results  of  a study  conducted  to  assess  the 
impact  of  the  Automated  Radar  Terminal  System  (ARTS  III)  upon  air 
traffic  control  system  productivity  and  capacity. 

In  December  of  1972,  the  FAA  Executive  Committee  directed  the  Associate 
Administrator  for  Engineering  and  Development,  in  coordination  with  the 
Associate  Administrator  for  Operations,  to  develop  a method  for  assessing 
the  impact  of  the  agency's  automation  programs  and  to  validate  FAA  and/or 
contractor  projections  on  productivity/capacity  increases.  A preliminary 
report,  issued  in  February  of  1974,  documented  the  results  of  a study 
designed  to  achieve  the  stated  purpose.  The  results  of  that  first  effort 
were  inconclusive,  and  it  was  decided  to  proceed  with  a follow-on  effort 
that  could  provide  a more  realistic  appraisal  of  the  impact  of  the  ARTS  III 
system. 

IVo  sets  of  data  were  established  to  provide  a base  from  which  conclusions 
could  be  drawn  as  to  the  type  and  degree  of  impact  of  automation.  The 
first  set  contained  workload  data  measured  at  two  TRACONS;  San  Antonio, 
Texas  (SAT),  and  San  Francisco/Oakland,  California  (BAY)  prior  to  the 
implementation  of  the  ARTS  III  system.  The  second  set  of  data  contained 
the  same  type  of  workload  data  measured  at  the  same  facilities  after  the 
ARTS  III  system  had  been  operational  for  a reasonable  period  of  time. 
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The  methodology  utilized  in  establishing  the  data  bases  included  measuring 
the  workload  at  the  various  operating/control  positions.  This  was 
accomplished  by  identifying  workload  indicators  that  could  be  readily 
distinguishable  and  recordable.  In  addition  to  the  workload  indicators, 
other  relevant  information  was  also  recorded  and  utilized  in  the  analysis. 
This  information  included  traffic  volume  and  distribution,  staffing, 
weather  conditions,  airport  and  equipment  operational  status,  and  en 
assessment  of  the  controller's  degree  of  busyness,  expressed  as  a qualitative 
"pace  rating".  These  "pace  ratings"  considered  workload,  stress,  complexity, 
and  ranged  from  very  light  to  very  heavy. 

Analysis  of  the  data  included  the  comparison  of  measured  workload  and 
traffic  activity  at  specific  "pace"  levels.  These  comparisons  were  made 
on  a position  by  position  basis,  comparing  the  "before"  and  "after"  data 
sets,  to  determine  the  relationships  and  changes  induced  by  the  introduction 
of  the  ARTS  III  system  into  the  air  traffic  control  operation. 

The  results  of  the  analysis  indicate  that  the  ARTS  III  system  has  reduced 
workload  in  the  system  and  improved  productivity  and  capacity.  Equivalent 
comparisons  of  the  ARTS  III  and  non-ARTS  III  systems  reveal  that  these 
two  TRACON  facilities  experienced  a 10.5%  increase  in  capacity  (i.e., 
numbers  of  aircraft  handled  under  an  average  work  pace).  A reasonable 
estimate  of  the  productivity  increase  is  8.5%.  This  reflects  the  degree 
of  influence  attributable  to  ARTS  III,  by  itself,  since  the  only  environ- 
mental difference  between  the  two  sets  of  data  is  the  introduction  of  the 
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ARTS  III  system  into  the  air  traffic  control  operation.  All  other  pertinent 
areas  remained  the  same  (airspace  configuration,  operational  procedures, 
letters  of  agreement,  etc.).  Some  operating  position  designations  were 
changed,  but  this  was  a name  change  only  and  had  no  influence  on  the 
operation  of  the  air  traffic  control  system  within  that  particular  airspace. 

Analysis  of  individual  controller  workload  indicators  compared  at  the 
"average"  pace  level  reveals  the  following: 

• In  general,  the  volume  of  air/ground/air  transmissions  increased 
slightly  at  both  locations  after  the  ARTS  III  system  became 
operational.  While  verifications  of  aititude,  altitude  control 
instructions,  verifications  of  speed,  speed  control  instructions, 
etc.,  had  decreased  at  both  locations,  traffic  advisories  and 
other  control  instructions  (weather  and  vectors  to  expedite 
traffic  movement)  had  increased. 

• Interphone  activity  increased  slightly  (+2.1%)  at  SAT  after  the 
ARTS  III  system  became  operational,  but  decreased  (-12.6%)  at 
BAY.  Actions  requiring  coordination,  except  for  hand-off  of 
aircraft,  were  responsible  for  the  slight  increase  of  interphone 
activity  at  SAT. 

• At  BAY,  coordination  activity  via  interphone  increased  slightly. 
However,  interphone  activity  affecting  aircraft  hand-offs  and 
general  information  decreased  significantly. 

• At  SAT,  flight  strip  activity  and  oral  communications  had  increased 
in  volume.  This  is  due  to  the  fact  that  the  SAT  data  revealed  a 
significant  increase  in  total  number  of  "aircraft  minutes" 
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(duration  of  aircraft  time  under  actual  control)  and  a signifi- 
cant increase  in  "peak  aircraft"  handled  (highest  number  of 
aircraft  under  control  at  any  instant) . 

• At  BAY,  however,  the  data  reflected  a decrease  in  the  flight  strip 
and  oral  communication  activity.  This  is  due  to  the  fact  that  the 
total  number  of  "aircraft  minutes"  and  "peak  aircraft"  had  decreased. 

• Keypack  activity  (to  initiate/accept  aircraft  hand-offs;  start/drop 
tracks  on  aircraft;  quick-look,  etc.)  was  a new  workload  indicator 
imposed  by  the  ARTS  III  system.  The  reflected  impact  on  controller 
workload  is  an  additional  2.25  activities  per  5 minute  period. 

In  general,  it  is  the  opinion  of  the  study  team  that: 

1.  It  is  necessary  to  analyze  the  impact  of  the  ARTS  III  system  on 
each  specific  workload  indicator.  This  methodology  identifies 
those  areas  that  are  affected,  and  to  what  degree.  This  approach 
also  provides  guidance  in  the  planning  of  future  systems  and 
enhancement  packages,  identifying  areas  of  concern  and  areas 
requiring  some  measure  of  improvement. 

2.  After  careful  analysis,  it  is  clear  that  the  ARTS  III  system 
has  increased  the  productivity  and  capacity  of  the  terminal 
operation  portion  of  the  air  traffic  control  system.  Primary 
supporting  evidence  is  the  fact  that,  in  the  ARTS  III  environ- 
ment, the  air  traffic  controllers  handled  a greater  number  of 
aircraft  at  the  same  work  pace  than  had  been  previously  recorded. 
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1.  Introduction 


1. 1 Foreword 

This  report  documents  the  results  of  an  in-house  study  conducted 
from  August  1973  through  April  1976.  The  study  was  directed  by  the 
Federal  Aviation  Administration  Executive  Committee  (EXCOM)  and 
conducted  jointly  by  the  FAA's  Systems  Research  and  Development 
Service  (SRDS)  and  the  Air  Traffic  Service  (AAT) . Air  Traffic 
Service  facility  personnel,  along  with  National  Aviation  Facilities 
Experimental  Center  (NAFEC)  personnel  assisted  in  the  data  collection 
and  data  preparation  phases  of  the  project. 

The  purpose  of  the  study,  as  identified  in  the  EXCOM  directive  dated 
December  1,  1972,  was  to  develop  alternative  methods  to  assess  the 
impact  of  the  agency's  automation  program  upon  air  traffic  controller 
productivity  and  to  validate  FAA  and/or  contractor  projections 
wherever  possible.  A report  was  to  be  prepared  and  submitted  for 
EXCOM  review  by  January  15,  1973. 

1 . 2 Study  Team  Organization 

The  responsibility  for  organizing  and  conducting  the  study  was 
originally  assigned  to  the  SRDS  Analysis  Division,  ARD-600,  and  later 
to  ATC  Systems  Division,  ARD-100.  Responsibility  for  operational 
support  was  assigned  to  the  Air  Traffic  Service's  Operations  Research 
Branch  (AAT-12) , which  enlisted  additional  support  from  their  ATC 
Systems  Program  Division  (AAT-100) . The  study  team  was  augmented 
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by  air  traffic  controllers  and  administrative  personnel  from  the 
air  traffic  control  facilities  visited  during  the  data  collection 
phase  of  the  effort,  and  by  controllers  and  data  reduction  personnel 
from  NAFEC. 

1 . 3 Background 

The  study  team  presented  a Prospectus  for  determining  the  impact  of 
automation  on  air  traffic  control  productivity.  The  Prospectus  was 
presented  to  the  Director,  SRDS , on  September  6,  1973.  The  Prospectus 
contained  various  plans  (including  a contingency  plan)  for  conducting 
the  study.  A plan  was  approved  by  the  Director,  SRDS,  in  coordination 
with  the  Director,  Air  Traffic  Service,  and  the  study  was  implemented 
immediately . 

The  study  team  established  a data  base  by  measuring  workload  at  four 
different  air  traffic  control  TRACON  facilities.  Two  of  these 
facilities  were  operational  with  ARTS  III  (Phoenix,  Arizona,  and 
Miami,  Florida).  Two  were  not  ARTS  III  facilities  (San  Antonio, 

Texas,  and  San  Francisco/Oakland,  California).  This  first  effort 
consisted  of  an  operational  comparison  offcthe  "without  ARTS  III" 
data  and  the  "with  ARTS  III"  data.  The  results  of  the  impact  of 
the  automated  ARTS  III  on  productivity  were  influenced  by  the  many 
variables  that  existed  in  the  operational  differences  at  the  four 
subject  TRACONS.  This  influence  distorted  the  percentage  of  change 
in  productivity  sufficiently  to  render  the  results  of  the  first  effort 
inconclusive.  A study  report  was  submitted  in  February  of  1974  and 
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a decision  made  to  limit  distribution  and  consider  a follow-on 


effort.  A determination  was  made  in  October  of  1974  to  proceed  with 
the  contingency  plan  contained  in  the  original  Prospectus. 

2.  STUDY  APPROACH 

2 . 1 Modification  of  Workload  Indicator  Codes 

The  data  base  established  from  pre-ARTS  III  workload  measurements 
recorded  at  San  Antonio,  Texas  (SAT),  and  San  Francisco/Oakland, 
California  (BAY),  TRACONS  was  reviewed  to  ensure  completeness  and 
validity.  The  codes  and  definitions  of  the  workload  indicators 
were  revised,  and  the  number  of  indicators  was  reduced  to  25  from  33. 

It  was  determined  that  several  indicators  were  either  indiscernible, 
or  not  utilized,  and  the  codes  and  definitions  were  adjusted  accordingly. 
(See  Appendix  A) 

2 . 2 Workload  Data 

To  completely  understand  the  air  traffic  control  functions  and  the 
responsibilities  of  each  operating  position,  the  organization  and 
layout  of  both  subject  facilities  were  studied.  To  obtain  a repre- 
sentative sample  of  the  total  facility  operation  and  workload,  a 
determination  had  to  be  made  as  to  what  and  how  many  operation 
positions  needed  to  be  measured. 

San  Antonio  TRACON  has  17  control  positions,  of  which  12  are  actual 
radar  control  positions.  Workload  data  was  recorded  at  7 radar 
control  positions  that  provided  the  most  representative  and  busiest 
level  of  activity. 
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A total  of  128  hours  and  55  minutes  of  workload  activity  was  recorded 
at  these  positions.  All  of  the  data  was  recorded  in  increments  of  5 
minute  samples  for  analysis.  There  were  a total  of  1,547  workload 
samples  recorded  at  SAT,  compared  with  1,189  workload  samples  from 
the  pre-ARTS  III  data  base. 

San  Francisco/Oakland  (BAY)  TRACON  has  28  control  positions,  of  which 
18  are  actual  radar  control  positions.  Workload  data  was  recorded  at 
10  radar  control  positions  that  were  most  representative  of  the  facility 
operation.  A total  of  175  hours  of  workload  activity  was  recorded  at 
these  positions  (or  2,100  workload  samples).  This  compared  to  1,369 
workload  samples  from  the  Pre-ARTS  III  data  base. 

3.  DATA  COLLECTION 

An  attempt  was  made  to  record  workload  data  during  the  busiest  periods 
for  the  operating  position  being  measured.  Facility  logs,  sector 
hourly  traffic  profiles,  air  carrier  schedules,  stored  flight  plan, 
etc.,  were  examined  to  identify  periods  of  peak  activity.  Additionally, 
data  collection  periods  were  scheduled  to  utilize  any  advantage  provided 
by  seasonal  trends,  weather  conditions,  etc.  Operating  positions  were 
eliminated  for  measurement  only  when  the  historical  data  indicated  a 
level  of  busyness  that  would  not  provide  meaningful  data. 

Data  was  recorded  at  the  operating  position  by  a two-man  observation 
team  of  air  traffic  controllers.  A "full  performance  level" 
controller  from  the  facility,  certified  to  operate  the  control  position 
being  measured,  recorded  data  related  to  sector  workload  volume  and  pace, 
giving  full  consideration  to  complexity  and  stress.  Pace  was  divided 
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into  seven  gradients  of  busyness  (See  APPENDIX  B) . The  controllers 
were  tested  for  validity  and  uniformity  of  pace  rating  assignments. 

In  addition  to  the  pace  rating  assignments,  the  facility  controller 
recorded  data  related  to  facility  status,  airport  runway  configuration, 
traffic  flow  and  distribution,  weather,  and  any  other  information  that 
may  have  influenced  the  pace  ratings  (See  APPENDIX  C) . The  pace  ratings 
were  assigned  in  increments  of  five  minute  periods.  This  time  frame 
was  established  because  of  the  often  short  durations  that  aircraft  were 
in  the  sector  and  under  the  jurisdiction  of  the  controller.  Additionally, 
it  was  a reasonable  retention  time  span  for  the  pace  rater. 

The  second  observation  team  member  was  a controller  from  the  FAA's 
National  Aviation  Facilities  Experimental  Center  (NAFEC) . This  observer 
recorded  sector  controller  activity  on  a kymograph  recorder  utilizing 
the  workload  indicator  codes  described  in  Attachment  A.  This  provided 
information  relative  to  length  of  activity  in  time,  each  division  on 
the  graph  representing  one  second.  (See  APPENDIX  D.) 


4.  DATA  REDUCTION 

Workload  data  from  both  sets  of  measurements  (before  and  after  ARTS  III 
at  San  Antonio  and  San  Francisco/Oakland)  were  coded  and  keypunched. 

The  data  was  merged  into  files  and  sorted  by  facility,  operating  position, 
and  staffing.  The  primary  mathematical  techniques  involved  in  the 
analysis  of  the  numerical  data  were  regression  analysis  and  computation 
of  means  and  standard  deviations.  The  regression  analysis  approach  was 
utilized  in  determining  which  of  the  various  independent  variables  of 
workload  and  aircraft  activity  have  a significant  impact  on  the  work 
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pace  of  the  air  traffic  controller.  Means  and  standard  deviation 
computations  were  made  in  order  that  comparisons  could  be  made  of 
the  effect  of  automation  upon  the  various  workload  indicators. 


5.  DATA  ANALYSIS 

5 . 1 Operating  Position  Designator  Changes 

Since  there  were  changes  in  operating  position  (sector)  designations 
at  both  TRACONs  (See  Appendix  E)  in  the  ARTS  III  configuration,  it 
was  necessary  to  assure  equivalence  of  the  two  sets  of  data  for 
comparison  purposes.  The  Standard  Operating  Procudures  (SOP's), 
letters  of  agreements,  and  other  pertinent  information  relative  to 
the  way  the  TRACONS  conducted  business  were  reviewed  to  determine 
that  the  duties  and  responsibilities  of  positions  of  operation  and 
allocated  airspace  and  functions  had  not  changed.  It  was  determined 
that  the  only  differences  were  in  name  designator  only. 

5 . 2 Def initions 

Peak  Aircraft  is  the  highest  instantaneous  aircraft  count  observed 
during  a five  minute  workload  sample. 

Aircraft  Minutes  is  the  total  number  of  minutes  that  all  aircraft 
were  under  jur idictional  control  of  the  operating  position  for  a 
workload  sample  period. 

Position  (Sector)  Flight  Time  is  the  average  number  of  minutes  that 
each  aircraft  was  under  the  jurisdictional  control  of  the  operating 
position. 
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Aircraft  Handled  is  a measure  of  aircraft  flow  rate  and  is  computed 


by  dividing  aircraft  minutes  by  Position  Flight  Time. 

5 . 3 Validation  of  Pace  Ratings 

The  pace  ratings  for  both  "Before"  data  sets  were  examined  to  determine 
which  rating  could  best  serve  as  an  indicator  of  productivity  and 
capacity.  The  workload  samples  were  tabulated,  and  it  was  determined 
that  the  "average"  work  pace  would  be  suitable.  The  pace  ratings 
were  normalized  by  setting  each  operating  position's  "A"  (average) 
pace  rating  to  one.  The  peak  aircraft  handled  for  all  other  paces  was 
then  expressed  as  a percentage  of  the  peak  aircraft  handled  at  the  "A" 
pace.  These  values  were  computed  for  each  position,  summarized  by 
facility,  and  are  shown  in  the  following  table  (TABLE  5-2). 

TABLE  5-2 

PACE  RATING  EVALUATIONS  (Based  on  Peak  Aircraft) 

Pace  Ratings 


Facility 

VL 

L 

A- 

A 

A+ 

H 

VH 

San  Antonio 

.39 

.53 

.78 

1 

1.16 

1.53 

1.75 

Oakland 

.36 

.54 

.76 

1 

1.18 

1.52 

The  above  figures  support  the  validity  and  accuracy  of  the  pace 
rating  techniques  utilized  at  each  of  the  facilities. 

5 . 4 Regression  Analysis 

Multiple  linear  regression  analysis  was  used  to  investigate  the  degree 
of  significance  that  each  of  the  workload  indicators  and  aircraft 
activity  variables  had  upon  determining  the  work  pace  of  the  air 
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traffic  controller.  The  initial  computer  runs  utilized  all  measured 
values  as  independent  variables,  and  pace  as  the  dependent  variable. 
Partial  correlation  coefficients  were  then  examined  to  determine  the 
closeness  of  relationship  between  pace  and  all  other  variables. 

Additional  regression  runs  were  then  made  using  only  those  variables 
that  had  some  relationship  to  pace,  i.e.,  a correlation  coefficient 
in  excess  of  0.50.  A summary  of  the  numerical  results  is  contained 
in  Appendix  F. 

Columns  3 and  5 of  Appendix  F indicate  the  percentage  of  the  variation 
in  the  dependent  variable  (pace)  that  can  be  attributed  to  the  linear 
variation  of  the  independent  variables.  For  the  "Before”  regression 
runs,  sector  flight  time  was  input  as  a constant  for  each  position. 

Since  equivalent  aircraft  is  calculated  by  dividing  aircraft  minutes 
by  sector  flight  time,  the  equivalent  aircraft  variable  is  not  linearly 
independent  and  could  not  be  included  in  any  regression  run  that  included 
aircraft  minutes.  Correlation  coefficients  for  equivalent  aircraft  would 
be  the  same  as  those  for  aircraft  minutes.  A review  of  the  significant 
variable  column  in  Appendix  F indicates  that  only  the  traffic  variables 
of  equivalent  aircraft,  aircraft  minutes,  and  peak  aircraft  are 
consistently  significant  for  all  positions  at  both  locations.  In  fact, 
just  using  these  indicators  and  the  regression  equations  derived,  it 
is  possible  to  predict  the  controller  work  pace,  within  one  pace  rating, 
over  80%  of  the  time. 
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The  lack  of  appreciable  correlation  between  pace  and  many  of  the 
workload  indicators  is  probably  due  to  the  nature  of  the  air  traffic 
control  process  and  related  environmental  factors.  That  is,  during 
low  traffic  periods,  the  controller  may  occupy  himself  by  giving 
advisories,  checking  of  speeds  and  altitudes,  or  otherwise  engaging 
in  "small  talk"  that  does  not  actually  increase  his  work  pace. 

During  high  traffic  periods,  the  controller  is  occupied  with  the 
mental  processes  of  traffic  planning  and  sequencing,  and  many  activities, 
if  not  absolutely  required  for  conflict  resolution  or  safety,  can  be 
omitted  or,  at  least,  deferred  for  some  period  of  time.  This  changing 
nature  of  the  controller's  activity  tends  to  distort  some  of  the 
linear  relationships  that  one  might  normally  expect  to  be  revealed  by 
the  use  of  multiple  linear  regression  analysis. 

5 . 5 Comparison  of  "Average"  (A)  Pace  Data 

The  "average"  (A)  pace  aircraft  activity  for  each  facility  was  computed 
for  the  before  ARTS  III  data  and  the  after  ARTS  III  data.  The  following 
table  (TABLE  5-3)  reflects  the  hourly  aircraft  handled  at  the  A pace, 
by  position  of  operation,  at  the  same  staffing,  before  and  after  ARTS  III. 

It  is  evident  from  the  comparison  of  the  two  sets  of  measurements  that 
tnere  is  a definite  increase  in  the  controller's  ability  to  handle 
more  aircraft  at  the  same  work  pace  in  the  ARTS  III  environment  than 
in  the  non-ARTS  III  operation. 
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TABLE  5-3 

SAN  FRANCISCO/OAKLAND  (BAY)  TRACON  ' 


OPERATING* 

POSITION 

STAFFING 

MEASURED  A/C  HANDLED  PER  HOUR 
AT  THE  AVERAGE  WORK  PACE 

BEFORE  ARTS  III  AFTER  ARTS  III  % CHANGE 

AR3 

1 

14.5 

16.1 

+11 

AR4 

1 

18.0 

21.0 

+17 

AR9 

1 

26.4 

25.9 

-2 

AR10 

1 

24.0 

28.0 

+17 

AR1 

1 

25.6 

29.0 

+13 

AR2 

1 

24.5 

26.0 

+7 

DR6 

1 

28.1 

23.8 

-15 

DR2 

1.5 

31.6 

36.5 

+16 

DR1 

1.5 

30.8 

30.5 

-1 

DR5 

1 

27.2 

22.3 

-18 

AVERAGE  INCREASE  PER  OPERATING  POSITION  = +4.5% 


SAN  ANTONIO,  TEXAS  (SAT)  TRACON 

MEASURED  A/C  HANDLED  PER  HOUR 
AT  THE  AVERAGE  WORK  PACE 

OPERATING* 

POSITION  STAFFING  BEFORE  ARTS  III  AFTER  ARTS  III  % CHANGE 


AR-2 

1 

14.5 

16.6 

+14 

AR-2 

1.5 

14.3 

20.6 

+44 

AR-3 

1 

15.5 

19.0 

+23 

AR-3 

1.5 

18.8 

21.0 

+12 

AR-5 

1 

12.0 

17.3 

+44 

AR-5 

2 

15.0 

19.7 

+31 

AR-6 

1 

27.4 

31.9 

+16 

AR-6 

1.5 

29.6 

34.7 

+17 

AR-7 

1 

23.9 

18.6 

-22 

AR-7 

1.5 

28.9 

26.4 

-9 

DR-1 

1 

18.4 

24.6 

+34 

DR-1 

2 

21.8 

24.1 

+11 

DR- 3 

1 

22.4 

22.2 

-1 

AVERAGE 

INCREASE  PER  OPERATING 

POSITION  = +16.5% 

*"Af  ter" 

position  designation 
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This  analysis  indicates  an  average  increase  in  capacity  for  the  two 
subject  TRACONS  of  10.5%.  There  is  no  doubt  that  this  increase  in 
capacity  is  attributable  to  ARTS  III  alone,  since  the  only  difference 
in  the  two  data  sets  was  the  introduction  of  the  ARTS  III  system 
itself. 

The  workload  indicators  were  compared  individually  in  order  to  determine 
what  impact  the  ARTS  III  system  may  have  had  on  each  particular  function 
performed  by  the  air  traffic  controller.  Appendix  G clearly  depicts 
the  areas  or  "work  functions"  performed  by  the  controllers  that  are 
affected  and  to  what  degree.  Additionally,  the  two  sets  of  data  were 
compared  in  an  attempt  to  determine  what  impact  the  ARTS  III  system  may 
have  had  on  the  manner  in  which  the  controller  does  business  or  if  it 
altered  the  manner  in  which  the  system  now  functions.  Since,  in  many 
instances,  the  frequency  of  occurrence  of  a particular  workload  indicator 
is  extremely  low,  it  was  necessary  to  group  certain  indicators  into 
broad  categories  to  be  used  for  comparison  purposes.  The  following 
paragraphs  present  the  findings  for  each  workload  category. 

Average  Sector  Flight  Time  was  reduced  at  both  TRACONS.  BAY  TRACON 
data  reflects  a decrease  of  12.8%  while  SAT  TRACON  decreased  0.9%. 

This  reduction  in  average  sector  flight  time  was  the  result  of  traffic 
being  expedited  through  the  sector  because  of  the  readily  available 
altitude  and  speed  information  displayed  continually  in  the  data  block 
on  the  radar  display,  thus  requiring  less  vectoring  and  other  control 
instructions  on  the  part  of  the  controller.  Earlier  hand-offs  of  the 
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flight  information  contained  in  the  data  block  also  caused  a reduction 
in  the  flight  time  through  the  sector. 

Control  Type  Mess  igc.  , transmitted  via  the  air/ground/air  facilities 
increased  in  frequency  at  both  TRACONS.  It  is  significant  to  note 
however,  that  air/ground/air  activities  related  to  speed  and 
altitude  control  and  verification  were  reduced  sharply  at  both  locations. 
For  example,  at  San  Antonio  the  altitude  verification  workload  indicator 
was  reduced  from  a five  minute  average  of  9.96  (total  of  all  positions) 
before  ARTS  III  to  4.87  after  ARTS  III.  At  San  Francisco/Oakland  the 
corresponding  figures  are  17.58  and  7.07. (See  Appendix  F)  . Speed 
control  instructions  were  reduced  by  similar  appreciable  percentages 
at  both  locations.  At  the  same  time,  other  air/ground/air  activity, 
such  as  vectors  to  shorten  flight  paths,  weather  and  traffic  advisories, 
etc.,  increased  at  both  TRACONS.  The  indication  is  that  the  automated 
system  has  reduced  the  "decision  making"  workload  functions  for  the 
controller,  thereby  increasing  his  ability  to  provide  additional  and 
better  service,  resulting  in  a safer  system. 

Coordination  and  Flight  Data  type  workload  activity  conducted  via 
the  interphone  system  decreased  12.6%  at  San  Francisco/Oakland  and 
increased  by  2.1%  at  San  Antonio.  Interphone  activity  related  to 
coordinating  and  affecting  hand-offs  of  aircraft  control  from  one 
operating  position  to  another  decreased  significantly  as  a result  of 
the  automated  hand-off  feature  of  the  ARTS  III  system.  Transmitting 
flight  data  information  from  position  to  position  was  also  reduced 
at  both  TRACONS  since  this  flight  data  information  is  displayed  on  the 
radar  screen  continually  (See  Appendix  G) . 
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Other  Communications,  such  as  direct  oral  and  visual  coordination 


requirements  were  reduced  at  both  TRACONS.  This  reduction  is 
attributable  also  to  the  data  block  feature  presented  on  the  controller's 
radar  display,  negating  the  need  to  communicate  via  other  means. 

Flight  Strip  activity  increased  in  frequency  of  occurrence  at  San 
Antonio.  This  activity  increased  by  17.4%,  from  42.49  before  ARTS  III 
to  49.91  after  ARTS  III  (total  of  all  positions).  This  increase  was 
in  proportion  with  the  increase  in  aircraft  handled  and  is,  therefore, 
self  explanatory.  At  San  Francisco/Oakland  the  flight  strip  workload 
activity  decreased  by  5.9%  while  the  equivalent  aircraft  handled 
increased  by  3.3%  (Appendix  G) . The  rate  of  change  at  San  Francisco/ 
Oakland  is  not  considered  significant  and  any  conclusions  derived  from 
this  information  would  be  speculative. 

Equipment  Adjustment  type  activity  (changing  brilliance,  focus  or 
contrast  on  radar  displays,  adjusting  ambient  lighting,  background 
lights,  etc.)  was  reduced  from  40%  to  50%  at  both  TRACONS.  This  was 
quite  noticable  at  both  locations,  and  particularly  in  view  of  the 
normal  tendency  to  experiment  or  play  with  new  gadgets.  Queries 
regarding  this  subject  at  both  TRACONS  resulted  in  replies  typical 
of  "the  equipment  is  working  beautifully,  installed  and  adjusted  well, 
no  need  to  touch  it!" 

Key pack  workload  activity  is  the  only  indicator  where  the  ARTS  III 
system  inflicted  a penalty,  at  both  TRACONS.  Since  this  type  of 
controller  workload  activity  was  non-existent  in  the  "before  ARTS  III" 
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operation,  it  was  expected  that  some  increase  in  workload  would  be 
evident.  At  San  Antonio,  the  workload  imposed  by  this  activity 
represents  an  average  of  2.08  additional  controller  actions  per  five 
minute  period.  At  San  Antonio,  the  figure  is  2.43  controller  keypack 
actions  per  five  minute  observation  period. 

6.  CAPACITY/PRODUCTIVITY  RELATIONSHIPS 

As  shown  earlier,  the  average  increase  in  capacity  for  the  two  subject 
TRACONS  is  10.5%.  If  one  wishes  to  consider  that  this  represents  an 
additional  10.5%  of  output  (aircraft  handled)  for  the  same  level  of 
input  (staffing),  10.5%  can  also  be  thought  of  as  the  potential 
productivity  increase.  However,  in  actual  practice,  this  productivity 
increase  cannot  be  realized  since  fluctuating  demand  levels  will  prevent 
a control  position  from  continuously  operating  at  capacity. 

Realized  productivity  gains,  then,  will  be  somewhat  less  than  10.5%. 

This  gain  could  be  estimated  by  referring  to  the  terminal  staffing 
standards  contained  in  Order  1380. 33A,  raising  the  staffing  break-point 
values  by  10.5%,  and  re-computing  staffing  requirements.  Staffing 
decreases  occur  when  a shift  in  the  break  point  causes  position 
staffing  to  be  reduced  from  1.5  to  1.0,  or  from  2.0  to  1.5. 

In  consideration  of  the  fact  that  an  ARTS  III  Staffing  Standard  Study 
was  conducted  by  Air  Traffic  Service  and  the  Office  of  Management 
Systems  in  1975,  this  approach  was  not  followed.  The  staffing  standard 
study  modified  the  rules  for  determining  position  staffing  in  that 
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average  position  flight  time  is  now  a part  of  the  criteria  (Notice 
N1380.67).  The  effect  of  this  change,  along  with  other  study  findings, 

V 

was  to  reduce  ARTS  III  staffing  by  approximately  8.5%.  This  figure  is 
considered  to  be  quite  compatible  with  the  capacity  figure  of  10.5%, 
and  is  a most  reasonable  estimate  of  the  ARTS  III  productivity  increase. 

7 . CONCLUS IONS 

It  is  the  opinion  of  the  study  team  that  all  workload  and  performance 
data  used  in  this  effort  and  obtained  through  actual  measurements  at 
the  operating  positions  in  the  two  subject  TRACONS  is  accurate  and 
representative  of  the  air  traffic  control  terminal  operation. 

After  careful  analysis,  and  ensuring  that  proper  consideration  was 
given  to  all  pertinent  facts,  it  is  clear  that  the  ARTS  III  system 
has  definitely  increased  the  productivity  and  capacity  of  the  terminal 
operation.  The  most  prominent  piece  of  evidence  that  substantiates 
this  statement  is  the  fact  that  the  air  traffic  controllers  did,  indeed, 
handle  a greater  number  of  aircraft  at  the  same  work  pace  than  had  been 
previously  recorded. 

The  methodology  employed  in  this  endeavor  identifies  in  every  detail 
those  areas  affected  by  the  introduction  of  the  ARTS  III  system  and 
to  what  degree.  This  is  especially  useful  in  planning  and  developing 
improved  systems  or  enhancement  packages. 
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The  results  of  this  effort  should  also  clarify  the  issue  and  question 
of  inducing  an  intolerable  "button-pushing"  workload  (via  the  keypacks) 
onto  the  air  traffic  controller.  It  is  evident  in  this  data  that 
that  is  not  the  case. 
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APPENDIX  A 


CODE.:  AND  DEFINITIONS  OF  CONTROT  TER  WORK  ACTIVITIES 


CODE 

TOR  productivity/capacity  study 

( BEFORE  ARTS  III) 

WORKLOAT  indicator 

ACTIVITY 

130 

131 

lUO 

1^1 

LOO 

UOO 

altitude  control  instruction 
altitude  verification 
speed  control  instruction 
.;peed  verification 
other  control  instruction 
advisory 

air/ground/air 

GHF 

RHF 

CHS 

RHS 

handoff  - to  another  facility 

handoff  - from  another  facility 

handoff  - to  complex  within  the  facility 

handoff  - from  a complex  within  the  facility 

interphone 

CC1 

CF 

;s 

I NT 

coordination  between  controller  8c  coordirator 
coord i ration  with  another  facility 
coordination  within  the  same  facility 
other  transactions 

interphone 

GHM 

RUM 

CC 

CR 

CH 

CFD 

give  a verbal  handoff 

receive  verbal  handoff 

coord irate  w/coordinator  position 

coord  irate  w/complex  outside  the  facility  position 

coordinate  w/radar  controller 

coord  irate  w/handoff  controller 

coord  irate  v/ flight  data  position 

oral 

KGH 

KRH 

KU 

KI 

KG 

quick  look 
initiate  handoff 
accept  handoff 
update/ change/ cancel 
request  information 
unknown  keyboard  action 

keypack 

CV 

coord iration 

visual 

FS 

all  flight  strip  activity 

manual 

M 

monitoring 

O 

stand  -by 

AE 

LC 

adjusts  radar,  radios,  etc. 

refers  to  darts,  maps,  handbooks,  etc.  A-l 

APPENDIX  A (continued) 


CODEC  AND  DEFINITIONS  OF  CONTROLLER  WORK  , 
FOR  productivity/capacity  STUDY 

ACTIVITIES 

CODE 

(AFTER  ARTS  III) 

WORKLOAD  INDICATOR 

ACTIVITY 

130 

131 
14  0 
141 
100 
4 00 

altitude  control  instruction 
altitude  verification 
speed  control  instruction 
speed  verification 
other  control  instruction 
advisory 

air/ground/air 

GHF 

RHF 

CHS 

RH3 

haidoff  - tc  another  facility 

handoff  - from  another  facility 

handoff  - to  complex  within  the  facility 

handoff  - from  a complex  within  the  facility 

interphone 

CCI 

CF 

C3 

1NT 

coordination  between  controller  and  coordinator 
coord i ration  with  another  facility 
coordination  within  the  same  facility 
other  transactions 

interphone 

GHM 

RHM 

give  a verbal  handoff 
receive  verbal  handoff 

oral 

OL 

KGH 

KRH 

KU 

KG 

quick  look 
initiate  handoff 
accept  handoff 
update/ change/ cancel 
unknown  keyboard  action 

keypack 

CV 

coordination 

visual 

FS 

all  flight  strip  activity 

manual 

AE 

LC 

adjusts  radar,  radios,  etc. 

refers  to  charts,  maps,  handbooks,  etc. 
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APPENDIX  B 


WORK  PACE  DEFINITIONS 

1.  Very  Light  Workload  - "VL".  A "VL"  rating  should  be  assigned  when 
the  work  pace  level  is  so  low  that  relatively  little  attention  has  to  be 
paid  to  the  position  of  operation.  Minimal  exertion  is  required. 

2.  Light  Workload  - "L".  An  "L"  rating  should  be  assigned  when  the  work 
pace  is  such  tnat  mere  than  minimal  exertion  is  required,  but  the 
complexity  of  situations  is  such  to  only  engage  the  controller's  complete 
attention  periodically.  There  are  no  complex  control  situations. 

3.  Average  Workload  - "A".  An  "A"  should  be  assigned  when  the  situation 
complexity  requires  almost  full  time  attention  of  the  controller.  The 
workload  is  evenly  distributed  and  places  no  unusual  demand  udoii  the 
controller.  This  pace  could  be  maintained  up  to  an  eight-hour  period 
with  normal  relief. 

a.  - Gradient.  A - should  be  assigned  when  significantly  less  than 
full  attentiveness  is  required  at  the  position;  the  demands  placed 
.upon  the  controller  are  slightly  less  than  one  could  expect  at 
average.  Infrequent  periods  of  inactivity  occur. 

b.  + Gradient.  A + should  be  assigned  when  the  demands  are  slightly 
greater  than  "A".  Rare  periods  of  inactivity,  full  attentiveness  to 
the  position  is  required.  A controller  could  be  exp'  cted  tc  work 
at  this  pace  up  to  six  hours  with  normal  relief. 

4.  Heavy  Workload  - "H".  An  "H"  rating  should  be  assigned  when  the 
complexity,  and  exertion  required  to  cope  with  the  situation  necessitate 
rapid  decisions;  there  is  constant  operational  activity-  Demands  placed 
upon  the  controller  exceed  those  of  a normal  cace.  A controller  could 
be  expected  to  securely  deal  with  this  level  of  work  for  up  to  three  hours 

5.  Very  Heavy  - "VH".  A "VH"  should  be  assigned  when  there  is 
continuous  laborious  activity,  superior  exertion  is  required  and  the 
rapidity  of  response  and  thinking  processes  are  critical.  There  are 
delays  in  acknowledging  demands  placed  upon  the  position.  A controller 
would  be  "pushed"  to  maintain  this  pace  for  one  hour. 
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APPENDIX  E 


OPERATING  POSITION  (SECTOR)  DESIGNATIONS 
SAN  ANTONIO  TRACON 


BEFORE  ARTS  III  AFTER  ARTS  III 


DR- 1 

DR-1 

DR- 2 

DR- 3 

AR-1 

AR-2 

AR-2 

AR-3 

AR-3 

AR-5 

AR-4 

AR-6 

AR-5 

AR-7 

AR-6 

R-l 

AR-7 

R-2 

SAN  FRANCISCO 'OAKLAND  T?_ACCN 


DR-1 

DR- 2 

DR- 2 

DR-1 

DR- 3 

DR- 5 

AR-1 

AR-3 

AR-2 

AR-4 

AR-3 

AR-9 

AR-4 

AR-10 

AR-5 

AR-1 

AR-6 

AR-2 

AR-8 

DR- 6 

E-l 


APPENDIX  F 


OAKLAND  (BEFORE) 
REGRESSION  ANALYSIS  SUMMARY 


POSITION* 

STAFFING 

R2 

(All 

Variables) 

SIGNIFICANT 

VARIABLES 

R2 

(Selected 

Variables) 

AR1 

1.0 

.771 

MIN, PK, A/C, 0/C, ADV, FS 

.691 

AR2 

1.0 

.774 

MIN,PK,A/C,0/C 

.714 

AR3 

1.0 

.586 

MIN, 0/C 

.532 

AR4 

1.0 

.767 

MIN,PK,A/C,0/C,FS 

.684 

AR9 

1.0 

.676 

MIN ,PK,A/C ,0/C ,FS 

.704 

AR10 

1.0 

.742 

MIN, PK, A/C, O/C, ADV, VC, FS 

.625 

DR1 

mm 

.529 

MIN, PK, A/C, A/V, 0/C, FS 

.493 

DR2 

1.5 

.565 

MIN, PK, 0/C 

.506 

DR5 

1.0 

.667 

MIN, PK, A/C, 0/C 

.587 

DR6 

1.0 

.688 

MIN, PK, A/C, 0/C 

.612 

*"After"  position  designator 


INDEX  OF  VARIABLES 

MIN  - Aircraft  Minutes 

PK  - Peak  Aircraft 

A/C  - Altitude  Control  Message 

A/V  - Altitude  Verification  Message 

0/C  - Other  Control  Message 

ADV  - Advisory  Message 

VC  - Visual  Coordination 

FS  - Flight  Strip  Activity 


F-l 


APPENDIX  F 


OAKLAND  (AFTER) 
REGRESSION  ANALYSIS  SUMMARY 


POSITION* 

STAFFING 

R2 

(All 

Variables) 

SIGNIFICANT 

VARIABLES 

R2 

(Selected 

Variables) 

AR1 

1.0 

.695 

EQ, MIN, PK, A/C, 0/C, ADV 

.614 

AR2 

1.0 

.622 

EQ,MIN,PK,0/C,ADV 

.577 

AR3 

1.0 

.725 

EQ,MIN,PK,A/C,0/C 

.589 

AR4 

1.0 

.708 

EQ,MIN,PK,A/C,0/C 

.541 

AR9 

1.0 

.486 

EQ,MIN,PK,A/C,0/C 

.618 

AR10 

1.0 

.599 

EQ,MIN,PK,0/C 

.563 

DR1 

1.5 

.499 

EQ,MIN,PK,0/C 

.442 

DR2 

1.5 

.500 

EQ ,MIN ,PK,0/C,FS 

.413 

DR5 

1.0 

.631 

EQ,MIN,PK,0/C 

.481 

DR6 

1 

1.0 

.672 

EQ,MIN,PK,0/C 

.644 

*"A£ter"  position  designator 


EQ  - Equivalent  Aircraft  Handled 

MIN  - Aircraft  Minutes 

PK  - Peak  Aircraft 

A/C  - Altitude  Control  Message 

0/C  - Other  Control  Message 

ADV  - Advisory  Message 

FS  - Flight  Strip  Activity 


F-2 


APPENDIX  F 


SAN  ANTONIO  (BEFORE) 
REGRESSION  ANALYSIS  SUMMARY 


POSITION* 

STAFFING 

R2 

(All 

Variables) 

SIGNIFICANT 

VARIABLES 

R2 

(Selected 

Variables) 

AR2 

1.0 

.807 

MIN, PK, A/C, A/V, 0/C, ADV 

.708 

AR2 

1.5 

.779 

MIN, PK, A/C, A/V, 0/C 

.771 

AR3 

1.0 

.502 

MIN, PK, 0/C 

.448 

AR3 

1.5 

.729 

. 

MIN, PK, 0/C 

.701 

AR5 

1.0 

.799 

MIN, PK, A/C, 0/C 

.729 

AR5 

2.0 

.777 

MIN, PK, A/C, 0/C 

.771 

AR6 

1.0 

.570 

MIN, PK, O/C 

.410 

AR6 

1.5 

.740 

MIN, PK, A/C, 0/C 

.615 

. 

AR7 

1.0 

.562 

MIN, PK, 0/C 

.532 

AR7 

1.5 

.596 

MIN, PK, A/C, O/C, ADV, GF 

.619 

DR1 

1.0 

.751 

MIN, PK, A/C, 0/C 

.705 

DR1 

2.0 

.805 

MIN, PK, O/C 

.783 

DR3 

1.0 

.738 

MIN, PK, O/C 

.735 

*"After"  position  designator 

INDEX  OF  VARIABLES 

MIN  - Aircraft  Minutes 

PK  - Peak  Aircraft 

A/C  - Altitude  Control  Message 

A/V  - Altitude  Verification  Message 

0/C  - Other  Control  Message 

ADV  - Advisory  Message 

GF  - Give  Handoff  to  Another  Facility 

F-3 


APPENDIX  F 


SAN  ANTONIO  (AFTER) 
REGRESSION  ANALYSIS  SUMMARY 


POSITION* 

STAFFING 

«2 

(All 

Variables) 

SIGNIFICANT 

VARIABLES 

R2 

(Selected 

Variables) 

AR2 

1.0 

.626 

EQ,MIN,PK,0/C 

.548 

AR2 

1.5 

.708 

EQ,MIN,PK,A/C,0/C 

.681 

AR3 

1.0 

.751 

EQ,MIN,PK,0/C 

.701 

AR3 

1.5 

.795 

EQ,MIN,PK,0/C,ADV,FS 

.786 

AR5 

1.0 

.781 

EQ,MIN,PK,0/C 

.759 

AR5 

2.0 

.205 

eq,min,pk,o/c 

.300 

AR6 

1.0 

.818 

EQ,MIN,PK,0/C,FS 

.773 

AR6 

1.5 

.763 

EQ,MIN,PK,0/C,FS 

.752 

AR7 

1.0 

.780 

EQ, MIN, PK, A/C, O/C, ADV 

.645 

AR7 

1.5 

.878 

EQ,MIN,PK,A/C,0/C,ADV,FS 

.775 

DR1 

1.0 

.760 

EQ,MIN,PK,A/C,0/C ,CF 

.639 

DR1 

2.0 

.570 

EQ,MIN,PK,0/C 

.537 

DR3 

1.0 

.668 

EQ,MIN,PK 

- 

.636 

*"After"  position  designator 

INDEX  OF  VARIABLES 


EQ  — Equivalent  Aircraft  Handled 

MIN  - Aircraft  Minutes 

PK  - Peak  Aircraft 

A/C  - Altitude  Control  Message 

0/C  - Other  Control  Message 

ADV  - Advisory  Message 

FS  - Flight  Strip  Activity 

CF  - Co-ordination  with  Another  Facility 
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APPENDIX  G 

XLAND,  CALIFORNIA  (OAK)  TRACON 
PACE  AVERAGES  PER  5 MINUTE  SAMPLE 
(BEFORE  ARTS  III) 
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APPENDIX  G 

OAKLAND,  CALIFORNIA  (OAK)  TRACON 
1 PACE  AVERAGES  PER  5 MINUTE  SAMPLE 
(AFTER  ARTS  III) 
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Cap  Change  X 14.1  44.5  22.5  11.5  44.0  31.2  16.7  17.0  -22.0  -8.7  34.0  10.4  -1.1  +214.1  16.47 

EQ.  A/C  Per  5 Minute  1.38  1.72  1.58  1.75  1.44  1.64  2.66  2.89  1.55  2.20  2.05  2.01  1.85  24.72  1.90 

EQ.  A/C  Per  Hour  16.56  20.64  18.96  21.00  17.28  19.68  31.92  34.63  18.60  26.40  24.60  24.12  22.20  296.64  22.82 


